Main Factors
Axis transformation: linear or log?
Data relationship: linear, polynomial, exponential, sigmoidal?
- Protocols:
- lineups - pick out the one that’s different in some way
- allows detection of differences in perception
- lots of validation of the method
- Aesthetics/setups:
- Compare two series
- prediction with dots - can they detect mismatch?
- prediction with dots + trendline - can they detect mismatch?
- you draw it - draw the next N observations in the series
- more relevant to what to expect from COVID-19 graphs
- doesn’t measure actual numerical understanding so much as visual prediction ability
- answer some question involving estimation/prediction
- requires more numerical understanding to translate the problem into a different response domain
- have to deal with rounding artifacts
The Problem with creating stimuli

With free y-axis scaling, it’s (relatively) easy to distinguish the shapes of various curves on a linear scale (quartic is included because it’s temporarily larger than exponential). However, once the log scale is introduced, it’s hard to distinguish the shape of the linear, quadratic, and quartic curves.
Other issues…
If we instead control the y scaling and scale x, you can actually distinguish the curves on the log axis, but not on the linear axis. The cognitive load of having to read the axes isn’t as overwhelming, though, because the scales line up vertically. 
If we instead scale y manually, things get a little more distinguishable… 
One solution
If we control the start and end points of each function, we can fit quadratic/quartic/linear/exponential functions to the points, solve for other parameters, and get reasonably flexible functions that have the same domain and range.

A better plan might be to just use Taylor expansion - saves us the trouble of fitting regressions…

The problem is that for odd-order approximations, y is negative for x < a (expansion center). That’s not super optimal… also, because we’re not fitting both endpoints, the ranges don’t match that well.
We can expand at a point slightly outside the domain, which results in more reasonable looking plots, but we’re still right back at the range problem. 
Lineup Options
Quadratic vs. Cubic (Plot #1)
Distinguishing between polynomial effects: 

Quartic vs. Exponential (Plot #1)


Quadratic vs. Exponential (Plot #1)

